Papers with unsupervised model
Unsupervised Morphology Learning with Statistical Paradigms (C18-1)
Copied to clipboard
| Challenge: | Existing models treat words as concatenation of morphemes, but some use transformations like rewrite rules to recognize dependencies between morphs. |
| Approach: | They propose an unsupervised model that exploits the notion of paradigms for morphological segmentation that can be applied to a homogeneous set of words. |
| Outcome: | The proposed model significantly improves on the Morpho-Challenge dataset in English, Turkish, and Finnish. |
A Simple Unsupervised Approach for Coreference Resolution using Rule-based Weak Supervision (2022.starsem-1)
Copied to clipboard
| Challenge: | state-of-the-art coreference models rely on labeled data, but an end-to-end model is needed to solve this problem. |
| Approach: | They propose an approach that leverages an end-to-end neural model in settings where labeled data is unavailable. |
| Outcome: | The proposed approach outperforms the previous best unsupervised model and outperformed the rule-based model on English OntoNotes corpus. |
ESTeR: Combining Word Co-occurrences and Word Associations for Unsupervised Emotion Detection (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies list as many as 154 human emotions, but most researchers agree on basic emotions such as anger, fear, disgust, sadness, surprise, and happiness. |
| Approach: | They propose an unsupervised model for identifying emotions using a novel similarity function based on random walks on graphs. |
| Outcome: | The proposed model can be computed efficiently and avoids dependence on labeled datasets. |
Tsetlin Machine Embedding: Representing Words Using Logical Expressions (2024.findings-eacl)
Copied to clipboard
| Challenge: | Embedding words in vector space is a fundamental first step in state-of-the-art natural language processing. |
| Approach: | They propose to embed words in vector space using propositional logic instead of dense vectors . they evaluate embeddings on intrinsic and extrinsic benchmarks and visualize word clusters based on their results . |
| Outcome: | The proposed model outperforms GLoVe on six classification tasks. |
Discovering Dialog Structure Graph for Coherent Dialog Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on dialog structure graphs from open-domain dialogs have limited number of dialog states and can be laborious and costly to annotate manually. |
| Approach: | They propose to use dialog structure graph as a model to discover hierarchical latent dialog states and their transitions from corpus to facilitate dialog management in a RL based dialog system. |
| Outcome: | The proposed model can discover meaningful dialog structure graph and significantly improve multi-turn coherence on two benchmark corpora. |
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document summarization use graphs and unlabeled documents . Existing models require labeled data, and it is expensive to create summarized documents. |
| Approach: | They propose to rank sentences using transformer attentions and pre-training objectives by unlabeled documents. |
| Outcome: | The proposed model achieves state-of-the-art on unsupervised summarization and is less dependent on sentence positions. |
Reinforcement Learning for Topic Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Topic modeling is a method to extract information from documents by grouping topics into topics and linking them with words describing them. |
| Approach: | They propose to replace the variational autoencoder with a continuous action space reinforcement learning policy and modify the neural network architecture to weight the ELBO loss. |
| Outcome: | The proposed model outperforms all other unsupervised models and performs on par with or better than most models using supervised labeling and contrastive learning. |
Open Domain Event Extraction Using Neural Latent Variable Models (P19-1)
Copied to clipboard
| Challenge: | Existing work on extracting events from news documents focuses on a set of pre-specified event types. |
| Approach: | They propose a latent variable neural model which is scalable to large corpus. |
| Outcome: | The proposed model performs better than the state-of-the-art method for event schema induction. |
Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing models that estimate annotators' reliability only consider binary labels and multi-class labels. |
| Approach: | They propose an unsupervised model which can handle binary and multi-class labels and integrate neural networks to model the dependency between latent variables and instances. |
| Outcome: | The proposed model can handle binary and multi-class labels and can estimate reliability of annotators across instances. |
Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train multiple languages with a shared encoder and multiple decoders are based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. |
| Approach: | They propose a multilingual unsupervised NMT scheme which trains multiple languages with a shared encoder and multiple decoders. |
| Outcome: | The proposed model performs better than the separately trained bilingual models on monolingual corpora and improves by 1.48 BLEU points on WMT test sets. |
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Question Answering (QA) is a field of increasing demand due to the availability of information online. |
| Approach: | They propose an unsupervised approach to training QA models with generated pseudo-training data by applying a simple template on a related sentence rather than the original context sentence. |
| Outcome: | The proposed approach improves the performance of a QA model on generated pseudo-training data. |
Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs (D19-1)
Copied to clipboard
| Challenge: | a novel approach to paragraph planning involves a high-level control of different levels of relations between sentences . a proposed model with both forms of relations outperforms baselines in partially conditioned paragraph generation task . |
| Approach: | They propose two models that integrate human-created and latent relations into document-level language models . they focus on paragraph-level plan between sentences to produce coherent text . |
| Outcome: | The proposed models outperform baselines in partially conditioned paragraph generation task. |
Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining (2022.acl-long)
Copied to clipboard
| Challenge: | a model trained to align only two languages can encode multilingually more aligned representations . a dual-pivot transfer theory is proposed for bilingual training . |
| Approach: | They propose methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts. |
| Outcome: | The proposed models reach the state of the art in unsupervised bitext mining and perform better than multilingually supervised models. |